Personalized multimedia services using a mobile service platform

ABSTRACT

A method for providing multimedia data from at least one controllable multimedia source to a mobile device includes providing a request path from the mobile device to a mobile service platform, receiving a request from the mobile device, obtaining a device profile from the mobile device, authenticating the identity of a user of the mobile device, and determining a user profile in response to the user identity. The method further includes authorizing control and access to the at least one multimedia source, providing a control channel from the mobile service platform to at least one multimedia server, providing multimedia data delivery information to the at least one multimedia server, and providing multimedia data to the mobile device in response to the request via the at least one multimedia server.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/136,540, filed May 1, 2002, which claims the benefit of U.S.Provisional Application No. 60/317,712, filed on Sep. 7, 2001 which ishereby incorporated herein by reference in its entirely.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

Not Applicable.

FIELD OF THE INVENTION

The present invention relates generally to communication systems and,more particularly, to mobile wireless communication and multimediadelivery systems.

BACKGROUND OF THE INVENTION

As is known in the art, wireless Internet access is different fromsimply accessing the Internet wirelessly. Mobile wireless users havedifferent needs, motivations and capabilities from typical wirelineusers. For example, a mobile user is usually in a multi-tasking mode,e.g., accessing the Internet while attending a meeting or shopping inthe mall. Typical Internet accesses are bursty in nature (checking stockquotes, weather, or finding a nearby restaurant) and task-oriented.Thus, browser-centric applications and elaborate user interfaces are oflimited utility since a mobile user usually carries small devices suchas a cell phone or a Personal Digital Assistant (PDA) having relativelysmall displays. These personalized devices, which are typicallyidentified by a wireless network address such as a cellular phonenumber, provide mobile users with continuous access to the Internet.

Advances in wireless networking and messaging technologies have givenmobile users many choices to access Internet contents and services.Existing devices and protocols include personal digital assistants(PDAs), such as Palm Pilots with Web Clipping, cell phones with wirelessapplication protocol (WAP) or short message service (SMS), e-maildevices, such as Blackberry and AT&T PocketNet, supporting Post OfficeProtocol 3 (POP3) and/or (Internet Message Access Protocol) IMAP, andAmerica On Line (AOL) Instant Messaging (AIM). In mobile applications,it is often desirable to receive multimedia data on mobile devices.

Multimedia data, for example, uncompressed video typically requirestransmission bandwidths of tens of megabits per second, and is thus notsuitable for bandwidth-limited environments. Even internationalstandards like ISO JPEG, MPEG1 and MPEG2 are also not well suited tosuch environments. However, recent compression standards, including ITUH.263, MPEG-4 and the in-progress H.26L are targeted towards“reasonable” quality bit rates as low as about 10 kbps. The standardsonly specify the structure of the bitstream, while leaving a great dealof flexibility in how an encoder creates a compliant bitstream. Tocontrol the bit rate and quality of the multimedia data, the encoder canvary processing of the signal prior to source coding, scale of thequantizer, mode selections and frame rate. The wireless multimediatranscoders (encoder/decoder pairs), for example conventional videotranscoders are designed to perform under the following constraints: 1)low spatial resolution, 2) low frame rate and 3) slowly moving andlow-detail contents. Some of the system design issues that arise in thewireless environment are data rate minimization, complexity, powerconsumption and reliability.

Implementation complexity and power consumptions are especiallyimportant in the design of portable multimedia terminals. The amount ofprocessing required for decoding, for example video data, is related tothe number of frames per second (frame rate) and the total number ofpixels in one frame (frame size). Therefore, the frame rate and theframe size need to be adjusted to the processing power of the clientdevice.

Wireless access links suffer from severe transmission conditions, suchas narrow bandwidth, higher bit error rates and high latency. One of thewireless environment's distinguishing characteristics is signal fadingdue to changes in the propagation path as the terminal moves andinterference from other terminals in the same cell or from neighboringcells. Reliability is measured by the bit error rate (BER). Typical BERson mobile wireless channels range from 10⁻² to 10⁻³. Thus achievingacceptable reliability requires powerful error detection and correctiontechniques which are continuously adapted to the changing transmissionconditions. Another problem with wireless links is congestion of thecontrol and request channels when these channels are used simultaneouslyto deliver the multimedia content.

It would, therefore, be desirable to provide personal multimediaservices delivered over a wireless communication channel to a variety ofmobile device types while minimizing congestion of the control andrequest paths. It would further be desirable to provide a mobile serviceplatform and separate multimedia servers having distinct channels fordelivering transcoded multimedia data and adapting the delivery of themultimedia data to fluctuations of the wireless communication channelconditions. It would also be desirable to enable a mobile user tocontrol multimedia sources.

SUMMARY OF THE INVENTION

The present invention provides a system for providing multimedia data toa mobile device including a mobile service platform for providingcommunication with a variety of protocols and devices. The mobileservice platform provides a message gateway for allowing mobile devicesusing a range of protocols and access networks to request delivery ofmultimedia data from a variety of multimedia sources and to control themultimedia sources. The mobile service platform controls a plurality ofmultimedia servers which provide separate delivery channels to avoidcongestion of the message gateway. Additionally, adaptive transcodersare located between the multimedia sources and the multimedia servers toprovide a stream of encoded multimedia data for delivery to the mobiledevices that adapt to fluctuation in the wireless delivery channel. Withthis arrangement, the mobile service platform manages the control path,while the multimedia servers handle content delivery. While theinvention is primarily shown and described in conjunction with portablemobile devices, it is understood that the invention is generallyapplicable to systems in which it would be desirable for differingdevice types and protocols to communicate with each other.

In one aspect of the invention, a method for providing multimedia datafrom at least one controllable multimedia source to a mobile deviceincludes providing a request path from the mobile device to a mobileservice platform, receiving a request from the mobile device, obtaininga device profile from the mobile device, authenticating the identity ofa user of the mobile device, and determining a user profile in responseto the user identity. The method further includes authorizing controland access to the at least one multimedia source, providing a controlchannel from the mobile service platform to at least one multimediaserver, providing multimedia data delivery information to the at leastone multimedia server, and providing multimedia data to the mobiledevice in response to the request via the at least one multimediaserver. With such a technique, personal multimedia services aredelivered over a wireless communication channel to a variety of mobiledevice types while minimizing congestion of the control and requestpaths, and a mobile user can control multimedia sources over thewireless channel. By routing the control paths through the mobileservice platform and the content delivery paths through multimediaservers, the control, transcoding, and multimedia delivery functions arehandled efficiently without overloading any particular communicationspipe. The inventive technique enables different modes of communicationfrom a multitude of handheld devices for efficient and personalizedmultimedia delivery.

In a further aspect of the invention, a method for providing multimediadata from a controllable multimedia source to a mobile device includesnegotiating a device profile, determining a plurality of encodingparameters, controlling a compressed video stream from the multimediasource, monitoring a decoded bit stream, measuring a transmissionparameter, and adjusting the outgoing frame rate, bit rate and framesize in response to the measured transmission parameter. With thistechnique, the delivery of the multimedia data continuously adapts tofluctuations of the wireless communication channel conditions.

In a still further aspect of the invention, a system for providingmultimedia data to a mobile device includes a mobile service platformhaving an authentication processor adapted to authenticate user commandrequests from the mobile device, and a multimedia source controlleradapted to receive the authenticated user command requests forcontrolling at least one multimedia source. The system further includesat least one multimedia server adapted to receive control commands fromsaid mobile service platform, and at least one transcoder adapted toreceive multimedia data from the at least one multimedia source andcoupled to the at least one multimedia server. Such an arrangementprovides a mobile service platform and separate multimedia servershaving distinct channels for delivering transcoded multimedia data.

In one embodiment, a mobile service platform provides personalizedmultimedia services, which enable a mobile user to remotely recordmultimedia programs, control cameras, and request the delivery ofpre-recorded or live multimedia content to his or her own mobile device.The mobile service platform authenticates users who send servicerequests from various mobile devices, directs a transcoder to transcodemultimedia content based on user, device, and transmission profiles, andauthorizes the delivery of multimedia content from a plurality of mediaservers to the proper mobile device over wireless LAN and CellularDigital Packet Data (CDPD) networks. The multimedia servers adaptautomatically to the fluctuations of the wireless channel conditions toprovide, for example, reasonable viewing of video on the mobile device.The mobile service platform essentially manages the control path, whilethe media servers handle the actual content delivery.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be more fully understood from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a schematic depiction of a system for providing multimediadata to a variety of mobile devices in accordance with the presentinvention;

FIG. 2 is a schematic block diagram of an exemplary architecture for thesystem of FIG. 1;

FIG. 3 is a schematic depiction of an exemplary communication pathconfiguration for a system for providing multimedia data to a variety ofmobile devices in accordance with the present invention;

FIG. 4 is an exemplary screen display for a mobile device controlling aVCR through AOL Instant Messenger in accordance with the presentinvention; and

FIG. 5 is a flow diagram illustrating the steps for providing multimediadata to a variety of mobile devices in accordance with the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

In general, the present invention provides personalized multimediaservice by integrating a mobile service platform, and a plurality ofmultimedia servers for wireless multimedia delivery. The mobile serviceplatform operates as a message gateway for allowing mobile devices usingvarious protocols on different access networks to access multimediaresources on the Internet and various other networks. The mobile serviceplatform includes a flexible architecture having a plurality ofcomponents that cooperate to service mobile device service requests.

Before providing a detailed description of the invention, it may behelpful to define some of the terms used in the description. As usedherein, the term “personalized multimedia service” refers to a servicethat enables a mobile user to remotely control multimedia sources, andrequest the delivery of multimedia content to a mobile device.Additionally, the service can provide access to personal “play lists” ofmultimedia content and authorized multimedia sources, and the servicecan adapt the format of the multimedia delivery to personal preferences.The multimedia service is further personalized by the customizedseamless delivery of related multimedia material based upon the mobileuser's profile.

Although the terms frame rate, frames per second, refresh rate, anddisplay size, generally refer to video data, as used herein these termsapply generally to other types of multimedia data such as audio, andreflect similar controllable transmission parameters such as the samplerate.

As described more fully below, authentication refers to the process ofverifying the mobile user's identity as service requests are receivedfrom various communication channels and authorization refers to theprocess of determining which users can control what multimedia sourcesand which users can request delivery of multimedia content from thesesources. As used herein, transcoding refers to the process of decodingand encoding multimedia content, transforming the original content fordelivery to mobile devices having different capabilities overcommunication channels having varying bandwidths, and adaptation refersto the process of handling dynamic fluctuations in wireless channelconditions by modifying the transcoding process.

FIG. 1 shows an exemplary embodiment of a personalized multimediaservices system 10 for enabling mobile users to receive multimedia datafrom a variety of multimedia sources using various protocols inaccordance with the present invention. The system 10 also enables themobiles users to control aspects of the operation of the multimediasources. The system 10 includes a mobile service platform 100 which canrun on a computer 102 having connections to a plurality of networks anddevices. The mobile service platform 100 further includes a wirelessdevice 104 for receiving and transmitting data via wirelesscommunication. The mobile phone device 104 can support Short MessageService (SMS) communication, which is well known to those skilled in theart. While shown as a wireless phone coupled to the computer, it will bereadily apparent to one of ordinary skill in the art that mobile serviceplatform functionality can be readily integrated into a single device.In addition, the mobile service platform 100 can include a number ofwireless devices, which can be the same or different type, coupled tothe computer 102. The system 10 further includes a plurality ofmultimedia servers 106 a-106 n (generally referred to as multimediaserver 106) which are adapted to receive multimedia data from aplurality of multimedia sources 108 a-108 m (generally referred to asmultimedia source 108).

It is understood that the mobile service platform 100 and the multimediaserver 106 can operate on a variety of known computers and operatingsystems, such as Unix and Windows. In one embodiment, the mobile serviceplatform 100 and the multimedia server 106 is implemented using the Javaprogramming language running in a Windows environment.

The mobile service platform 100 and the multimedia server 106communicate with various mobile devices and networks. In the illustratedembodiment, a cell phone 200 with two-way short messaging service (SMS),e.g., a Global System for Mobile Communication/Time Division MultipleAccess (GSM/TDMA) phone connected to a GSM/TDMA network 202, cancommunicate with the mobile service platform 100 through an SMS driverhosted on the mobile device server. Cellular Digital Packet Data (CDPD)devices 204, such as AT&T PocketNet phone 204 a and Palm V 204b, coupledto a CDPD network 206 can use a Wireless Access Protocol (WAP) gateway208 to access the mobile service platform 100 through the Internet 210.E-mail devices 214, such as a Blackberry mobile device, can use theStandard E-mail Protocol (SMTP) on the CDPD network 206 or a two-waypaging network 216 coupled to a mail server 218 to communicate with themobile service platform 100. The multimedia server 106 can delivermultimedia data to the mobile devices 200, 204 and 214 through the CDPDnetwork 206, the Wireless Access Protocol (WAP) gateway 208 and thepaging network 216.

In addition, PC device users and some PDAs can use AOL Instant Messenger(AIM) or web browsers to communicate with the mobile service platform100, which can support a Transmission Control Protocol (TCP) interface.Mobile devices can include an embedded module for communicating with themobile service platform 100 directly via the TCP interface.

The mobile service platform 100 can receive messages and commands fromthese devices, access Internet services and information on behalf of amobile user, and relay messages or Internet content back to the sendingdevices or other devices.

The mobile service platform 100 includes an architecture having aplurality of interface, logic, and access components that enable themobile service platform to communicate with a range of devices,protocols and information spaces. This arrangement hides the complexityof multiple devices and content sources from mobile users. The mobileservice platform 100 can include a proxy server that provides anenvironment for hosting agents and personalized services, which can beimplemented as reusable building blocks in the Java programminglanguage, for example. An exemplary proxy server known as iProxy, isshown and described in U.S. patent application Ser. No. 08/974,600,filed on Dec. 19, 1997, and 09/474,914, filed on Dec. 30, 1999, whichare incorporated herein by reference. In general, an iProxy agent, whichcan include a web-server, can be invoked like a regular common gatewayinterface (CGI) program. The iProxy system also allows scripts embeddedinside web pages to invoke agents to perform specialized processing. TheiProxy system maintains user profiles and adds intelligence to thetraditional HTTP proxy server to provide personalized, and value-addedservices such as filtering, tracking, and archiving.

FIG. 2 shows an exemplary architecture for a personalized multimediaservices system 300, which is similar to the system 10 of FIG. 1, havinga plurality of components that combine to provide a flexiblearchitecture that delivers multimedia data through a plurality ofmultimedia servers and interacts with a user of a mobile device througha separate mobile service platform to provide personalized services. Thesystem 300 includes a mobile service platform 340 having a personalinformation management processor 342 adapted to obtain information froma personal information database 344 and a user profile 346, a requestdispatcher 354 adapted to receive data requests and user commandrequests (collectively referred to as service requests) from amultimedia client 330 of a mobile device. The mobile service platform340 further includes an authentication processor 350 coupled to therequest dispatcher 354 and adapted to obtain personal information fromthe personal information database 344 and to provide information for theuser profile 346, an authorization processor 352 coupled to the requestdispatcher 354, a multimedia source controller 356 coupled to theplurality of multimedia sources 360 and an accounting processor 348.

The system 300 also includes a multimedia server 302 which includes atleast one multimedia transmitter 304, a feedback control processor 310and a session manager 312. The session manager 312 is coupled to theaccounting processor 348. The feedback control processor 310 accesses atransmission profile 306 and network state information 308. The system300 further includes a transcoder processor 320 (also referred to astranscoder 320) having an adaptive encoder 322 which is coupled to thefeedback control processor 310 and to the multimedia transmitter 304,and a decoder 324 which is coupled to one or more multimedia sources 360a-360 n (generally referred to as multimedia sources 360).

The system 300 further includes a multimedia client 330 having a deviceprofile 332, a multimedia decoder/player processor adapted to receivemultimedia data from the multimedia transmitter 304 over a wirelesslink, and a throughput manager 336 coupled over the wireless link to thefeedback control processor 310.

The blocks denoted “processor,” “platform,” “player,” “client,”“dispatcher” and “manager” can represent computer software instructionsor groups of instructions. Such processing may be performed by a singleprocessing apparatus which may, for example, be provided as part of asever, a mobile device controller, or set top boxes.

In one embodiment, mobile service platform 340 is, for example, similarto an iMobile system which is shown and described in more detail inpending U.S. patent application Ser. No. 10/037,570 which isincorporated herein by reference.

The multimedia server 302 and transcoder 320 automatically adapt themultimedia transmission to compensate for the fluctuations of thewireless channel conditions for maintaining a viewable display on themobile device. Each mobile device that communicates with the mobileservice platform 340 and multimedia server 302 registers its deviceprofile information with the mobile service platform 340. Deviceprofiles 332 include information for user devices, such as how muchinformation can be displayed. A device name is designated by protocoland account ID, i.e., protocol:acct_id. For example, an AIM user webciaois named aim:webciao. The mobile service platform 340 maintains adefault device profile 332 for each device type, and each mobile devicecan update that profile with device-specific information. The deviceprofile 332 can, for example, be a list of attribute-value pairs. Oneattribute is dev.format.accept, which determines what MIME type thedevice is allowed to accept. This information is used by the system 300in addition to the transmission profile 306 to transcode originalcontent to a format appropriate for this device, as described above. Thetransmission profile describes the protocol of the wireless channelenvironment. For example, Cellular Digital Packet Data (CDPD) or WLAN(802.11) protocols have substantially different bit rates and as aresult need media adaptation when traversing between network boundaries.The transmission profile can include a modeled transmission profile anda measured channel transmission profile. The measured channeltransmission profile can include an estimated available bandwidthquality of service measurement and a link delay measurement.

In general, each device is mapped to a registered user of the mobiledevice server. This mapping arrangement enables limiting access tolegitimate users of the mobile device server, and personalizing aservice based on the user profile. An illustrative device-to-user mapstored in the user profile 346 for a particular user on the mobileservice platform 340 is set forth below:

sms:+886935551826=herman

sms:+19085556842=chen

mail:dchang(research.att.com=difa (an alias)

aim:webciao=chen.

It is understood that the mobile service platform 340 of the presentinvention can rely upon a variety of authentication techniques. Sincethe mobile service platform 340 interacts with multiple networks andprotocols, the server relies on different authentication mechanisms. Inone embodiment, the mobile service platform 340 uses the cell phoneidentification on wireless phone networks, AOL buddy names on the AIMnetwork, and generic user ID and password information for WAP, HTTP, andtelnet clients. However, the mobile service platform 340 does not havecontrol over the security afforded by some of these networks.Alternative embodiments can include the SSH Secure Shell to provideend-to-end authentication services. In general, the technique used bythe mobile service platform 340 to authenticate a mobile user depends onthe device or protocol used. Trusting wireless networks, such asVoicestream/GSM and AT&T TDMA networks, to provide the correct cellphone id when a short message (SMS) is received is generally acceptableunless a cell phone is stolen and the user did not lock the phone with asecurity password. The mobile service platform 340 can also generallytrust the AOL network authentication for non-critical services. Userauthentication through the mobile service platform 340 itself isrequired if the user accesses the mobile service platform 340 throughtelnet, WAP, or HTTP. The following is an example of a user profile:

name=Robin Chen

password=xf2gbH3

default=$mail.1

# my addresses

sms.1=sms:+19085556842

mail.1=mail:chen@research.att.com

mail.2=mail:imobile@mobile.att.net

mail.all=$mail.1,$mail.2

aim.1=aim:webciao

# command aliases

sms.cmd.q=quote

sms.cmd.sn=sitenews

# address aliases

sms.addr.cc=aim:chrischen.

Where sms.1 is an sms identifier, and is here a phone number;

sms.cmd.q=quote indicates that q is an assigned shortcut for quotes; andcc is an assigned shortcut for the AOL instant messenger ID chrischen.

The user profile 346 stores the user name, password, and a list of thedevices that the user registers with the mobile device server. It alsostores command and address aliases. When a user accesses the mobileservice platform 340 through AIM using the id webciao, the mobileservice platform 340 determines from the user-device map that the useris chen and uses the user profile chen.ini for all later servicerequests from this device. For example, the following short message sentfrom a GSM phone: “forward $mail.1 q T” is interpreted as “forwardmail:chen@research.att.com quote T” according to the user profile. Thespecial character “$” requests that the mobile service platform 340 mapthe named device, i.e., mail.1, to its corresponding entry in theprofile.

The transmission profile 306 includes for example entries specifying theprotocol of the wireless channel in the following format:

dev.txprofile=wlan; or alternatively

dev.txprofile=cdpd;

where wlan indicates a protocol such as 802.11; and

cdpd indicates a Cellular Digital Packet Data protocol.

In operation, the multimedia client 330 is started on a mobile devicesuch as an iPAQ mobile wireless device manufactured by Compaq ComputerInc., and contacts the mobile service platform 340 to identify themobile device with the corresponding multimedia client 330. This can bedone in several ways. For example, the iPAQ supplies an IP address ofthe mobile device. There is a separate mechanism to keep track of whichIP addresses belong to which mobile server platform 340 user. Themapping, for example, is stored in the iMobile user-device map:ip:135.22.102.22=chen.

Attributes of this iPAQ device/multimedia client can be specified in thedevice profile. For example,

dev.format.accept=video/h.263; indicating that the device accepts theH.263 protocol; and

dev.size=128×96; indicating the display size of the device.

Upon activation, the client proxy negotiates its device profile with themobile service platform 340. The device profile includes the displaysize, audio-visual decoding capability and the CPU processing power.

The system 300 is designed to provide personalized mobile services withthe help of the personal information management processor 342 whichretrieves personal information such as user preferences from thepersonal information database 344. The transcoder processor 320 appliescontinuously updated transcoding to each multimedia stream enablingtransmission under different communication operating conditions.

When the multimedia client 330 is ready to retrieve a pre-recorded videosuch as cnn-news, the mobile service platform 340 examines the deviceprofile 332 and transmission profile 306 and determines that transcodingfrom MPEG2 to H.263 on the VCR server is necessary. The mobile serverplatform 340 then instructs the VCR server to move the transcodedcontent to the multimedia server and instructs the multimedia server todeliver the content to the multimedia client at IP address135.22.102.22.

In one embodiment, transcoding using the International TelecommunicationUnion (ITU) H.263 video compression standard for enhanced referencepicture selection mode is performed on each MPEG2 file right after it isrecorded. H.263 is a popular format used by wireless multimedia devices.Transcoding includes decoding the MPEG2 file and re-encoding the datausing H.263. In this embodiment, the transcoder uses an MPEG2 decoderbased on the MPEG Software Simulation Group (MSSG), a resolutiontransformer, transforming from varied picture size to quarter commonintermediate format (QCIF) with smoothing.

The multimedia client 330 decodes the incoming multimedia stream andforwards it to the display unit. Typical display devices include but arenot limited to handheld computers (Windows CE environment), personaldigital assistants (e.g. Palm Pilots) and multimedia phones. Themultimedia client 330 may also include private labeling or injectannotations to the decoded stream. Monitoring the decoded bit stream isaccomplished at the application layer. The multimedia client 330throughput manager 336 maintains a running average of the streamed bytesand measures the download throughput. The throughput manager 336notifies the feedback control processor 310 in the multimedia server 302of the current throughput. In response to the throughput measurement andthe network state information 308, the feedback control processor 310commands the transcoder 320 to adjust the outgoing frame rateaccordingly. The multimedia client 330 is also responsible forcollecting various profile information and sending the information tothe mobile service platform 340.

The multimedia server 302 includes the session manager 312 whichcontrols the generation of the compressed multimedia stream from variousmultimedia sources. The feedback control processor 310, based onmeasured throughput feedback, decides on a set of encoding parametersappropriate for each user. In an embodiment in which video data isdelivered to the mobile device these parameters include the encoding bitrate, the frame format, the frame size and the quantization table. Itwill be appreciated by those of ordinary skill in the art that similarencoding parameters including compression ratios are adjusted for otherforms of multimedia and protocols. Examples include image media such asGIF, JPEG and PNG; audio media such as Real Audio, wav, au; and videofiles such as Quicktime, MPEG, and Motion JPEG.

The encoding bit rate parameter acts as a reference for the transcoder.The encoding parameters supplied to the adaptive encoder 322 arecontinually adjusted such that deviation between the output frame rateand the reference frame rate is minimized. For video data, the frameformat switching is determined by estimating the display size and theprocessing power of the client device. By adaptively encoding themultimedia data, the system performance degrades gracefully in thepresence of errors, unexpected changes in operating conditions such asoverload, network congestions or abrupt variations in the end-to-endthroughput due to client mobility.

Further details on a specific implementation of an exemplary multimediaserver (referred to as iVideo) can be found in S. John, R. Jana, V.Vaishampayan, A. Reibman, “iVideo—A Video Proxy for the MobileInternet”, Proc of IEEE 11^(th) International Packet Video Workshop,Korea, May 2001.

FIG. 3 shows a plurality of data and control pathways in a communicationpath between mobile devices and an exemplary personalized multimediaservices system 400 in accordance with the present invention. The system400 includes a plurality of multimedia clients 410 a-410 n (generallyreferred to as multimedia clients 410) which operate on the mobiledevices (not shown) and communicate with a plurality of multimediaservers 412 a-412 n over wireless channels 420 a-420 n respectively. Themultimedia clients 410 communicate with a mobile service platform 460over wireless channels 426 a-426 n and control a variety of multimediasources, for example, a multimedia source 450 a, here a source of liveTV channels, a multimedia source 450 b, here a VCR server, andmultimedia source 450 n, here an X10 server, over logical channels 428a-428 n (generally referred to as logical channels 428). The logicalchannels 428 b and 428 n operate over physical channels 462 from themobile service platform 460 to the multimedia sources 450 b and 450 n.The mobile service platform 460 is coupled to the multimedia servers 412over communications channels 430 a-430 n. The plurality of multimediaservers 412 a-412 n are coupled to the corresponding plurality ofmultimedia sources 450 a-450 n over physical links 432 a-432 nrespectively. Additionally, the logical channel 428 a used forcontrolling the multimedia source 450 a can be supported over thephysical link 432 a from the multimedia server 412 a to multimediasource 450 a.

The system 400 also includes a plurality of transcoders 414 a-414 n(generally referred to as transcoder 414) coupled between the multimediasources 450 and the corresponding multimedia server 412. As a functionof the multimedia source and a network configuration, the transcoder 414can be a stand-alone transcoder, for example, transcoder 414 a coupledto multimedia server 412 a over channel 422 and coupled to themultimedia source 450 a over channel 434. The transcoder 414 can alsobe, for example, a transcoder 414 b which is integrated with themultimedia source 450 b. Finally the transcoder 414, can be integratedinto the multimedia server 412, for example, multimedia server 412 nincluding transcoder 414 n which is coupled to multimedia source 450 nover the channel 432 n. Multimedia source 450 n is coupled to aplurality of video cameras 470 a-470 n over channels 472 a-472 nrespectively.

It will be appreciated by those of ordinary skill in the art thatparticular channels described above in certain embodiments are wirelesschannels and in alternate embodiments are hard wired channels using oneor more communications protocols and networks.

As described above, the location of transcoders 414 are arranged toprovide efficient use of hardware and network bandwidth resources. Thetranscoders 414 are arranged as a function of supported media types anduser preferences based on the client decoding capabilities, and aregenerally located either within a multimedia server 412, as astand-alone resource, or within a multimedia source, for examplemultimedia source 450 b.

The mobile service platform 460 controls aspects of the operation of themultimedia servers 412 over communications channels 430 a-430 n. Theprotocol used for communication between the mobile service platform 460and each of the multimedia servers 412 can include a proprietaryprotocol that uses signaling based on TCP/IP communications.Alternatively a standards based signaling scheme such as the mediagateway control protocol, (MGCP) or real time control protocol (RTCP)

The multimedia servers 412 provide a channel 432 to the multimediasource and separate channels 420 to deliver multimedia content to themultimedia clients 410 while adaptively controlling the datatransmission to adjust automatically to wireless transmissionconditions. By using channels 420, congestion on the request channels426 from the multimedia clients 410 to the mobile service platform 460is minimized.

Now referring to FIG. 4, in which like reference numbers indicate likeelements of FIG. 3, an illustration of a display of a mobile device,shows control of a multimedia source 450 b, here a VCR server whichincludes transcoder 414 b, here an MPEG-2 encoder. The multimedia source450 b is coupled to multimedia server 412 b which is coupled to amultimedia client. The multimedia client 410 a has a daemon (backgroundsoftware module) which communicates with the mobile service platform 460to accept control commands in response to service requests. From themobile user's point of view, the multimedia content is delivered by themultimedia server 412 b operating as a controllable VCR connected to anetwork. The mobile user can change the channel, record the multimediastream, and select a recorded multimedia stream for display. In anembodiment where the system 400 includes an iMobile device server, theseservice requests can be issued through a new iMobile infolet called“vcr.”

After authentication over channel 426 a to the mobile service platform460, the mobile user controls the recording of video from the mobiledevice. The mobile service platform 460 determines which multimediasources can be controlled by which users. If a multimedia recordingserver at port 50000 of a machine at 135.207.19.231 is operated for aparticular user “chen,” then the following information is registered inhis user profile before mobile server platform 460 issues the recordingcommand through a TCP connection to the multimedia server:

video.vcr=vcr://135.207.19.231:5000The user can then send the following vcr requests:

vcr channel 33

vcr record cnn-news 20

These user command requests instruct the VCR server 450 b to switch tochannel 33 (the CNN channel on this VCR) and record 20 seconds of theprogram and store it in the file called cnn-news in the standard MPEG2format under the user's space. Here, the MPEG2 file has to be compressedto a suitable media format for delivery to the end user. The controlsignals issued from the end user travel via the mobile service platform460 to the VCR server 450 b. In this example, transcoding is needed totranscode MPEG2 to MPEG4 or other media types based on personalpreference or the multimedia client's 410 a decoding capabilities.

FIG. 4 shows these remote control commands during an instant messagingsession, which can easily run on most PDA's with wireless modems or AOLmobile communicator, a two-way pager, for example, a Blackberry e-maildevice such as the Research In Motion RIM 950. The first instant messageconfirms that the user is allowed to control that multimedia server. Theother two instant messages send the service requests.

In one particular embodiment, the multimedia source 450 a includes anATI All-in-Wonder video card which captures TV programs and has anoutput connected to the input of a transcoder 414 a, here a FutureTelMPEG-2 encoder. The control of the TV tuner is implemented by simulatingkeyboard events over logical channel 428 a. For the control of the MPEGencoding process, a PrimeWare USDK is used to control the transcoder 414a. Multimedia data streams are stored in memory buffers in thetranscoder and sent to the multimedia server 412 a remote machines forstreaming to the multimedia client 410 a. The multimedia data streamscan also be saved to local disks situated at the multimedia server. Amobile device user can specify the channel, starting time, and durationof the TV program. The multimedia server 412 can save the multimediastream into a file and the mobile device user can later retrieve anddisplay the multimedia content on mobile device.

Similarly, if a user would like to control X10 cameras in his backyard,the mobile service platform 460 needs to know that the user isauthorized to control devices connected to the X10 server of that housethrough the X10 home network. If the X10 server is listening to themobile service platform 460 at port 1010, the following should berecorded in the user profile:

x10.house.1=x10://135.207.25.147:1010

Several X10 cameras can be used on the X10 home network, but each X10video receiver can only receive one video source at any point of time toavoid signal interference. Each X10 camera can be addressed by a letterid followed by a numeric id. For example, two X10 cameras (a4, a7) cansend their video signals to the same X10 receiver. This set can bedefined in the user profile as “set.cam” by the following:

x10.set.cam=a4,a7

In this example, the camera a4 views a front door while the camera a7looks at a garage, the mobile user can switch between either one bysimply sending the following requests:

rx10 cam a4 on

rx10 cam a7 on.

The first request “rx 10 cam a4 on” causes mobile service platform 460to contact the x10 server running at the user's house, turns on cameraa4, and turns off all other cameras in the same group that share thesame receiver. Once a video source such as a recorded TV program or alive X10 camera becomes available, mobile server platform 460 can signala multimedia server 412 n to deliver the video in the right format astranscoded by transcoder 414 n to an authorized mobile device'smultimedia client 410 n over wireless channel 420 n.

In one particular example, a mobile user in Seattle is able to controlcameras and video recording in New Jersey and view a remote office at2-3 frames per second live using an iPAQ and a CDPD modem. In anotherexample, over a wireless LAN environment based on the 802.11b protocol,a mobile device can receive video at 20 frames per second or above on asimilar iPAQ with a WaveLan modem from Lucent.

Now referring to FIG. 5 in which like reference numbers indicate likeelements of FIG. 2, a flow diagram illustrates an exemplary process fordelivering multimedia content over separately controllable multimediaservers to a mobile device in accordance with the present invention. Itis understood that the rectangular elements are herein denoted“processing blocks” (typified by element 502 in FIG. 5) and representcomputer software instructions or groups of instructions. The diamondshaped elements in the flow diagrams are herein denoted “decisionblocks” (typified by element 538 in FIG. 5) and represent computersoftware instructions or groups of instructions which affect theoperation of the processing blocks. The flow diagrams do not depict thesyntax of any particular programming language. Rather, the flow diagramsillustrate the functional information used to generate computer softwareto perform the required processing. It should be noted that many routineprogram elements, such as initialization of loops and variables and theuse of temporary variables, are not shown. It will be appreciated bythose of ordinary skill in the art that unless otherwise indicatedherein, the particular sequence of steps described is illustrative onlyand can be varied without departing from the spirit of the invention.

At step 502, the mobile service platform 340 (FIG. 2) receives a requestfrom mobile device over a wireless request path. The request can beformatted, for example, using an HTTP protocol; an instant messagingprotocol, a WAP protocol; an SMS protocol, an e-mail protocol, a userdefined control signal transported on a TCP/IP protocol, and a userdefined control signal transported on a UDP/IP protocol format. Therequest can include a request for data delivery and can optionallyinclude a request to control a multimedia source.

At step 504, the mobile service platform 340 obtains a device profilefrom the mobile device. The profile is provided from a user agent of themultimedia client 330, for example, a mobile device microbrowser agent.The mobile service platform 340 device also supports a W3C standardcomposite capability preference profile (CC/PP) which facilitates theexchange of device capabilities with the mobile service platform 340. Aspart of a user provisioning process, i.e., the new user installationprocess, each user specifies the device type of the mobile device andthe “default” transmission protocol, i.e., CDPD or 802.11, of the mobiledevice.

At step 506, the mobile service platform 340 authenticates the identityof a user of the mobile device. The authentication can be accomplished,for example, by using a password, a biometric, or other means which areknown in the art.

At step 508, the mobile service platform 340 retrieves a user profile inresponse to the user identity from the personal information database.The user profile provides information which allows the delivery ofpersonalized services, for example, multimedia source preferences,multimedia play lists, and downloadable video clients for a particularmultimedia format.

At step 510, the mobile service platform 340 authorizes control andaccess to one the multimedia source if the user has permission toretrieve data and control the data source. At step 512, the mobileservice platform 340 validates and dispatches the user request. In oneembodiment in which the mobile service platform 340 includes the iMobiledevice server, and the device server invokes applets and infolets toprocess the request. At step 514, the mobile service platform 340allocates a multimedia server 302. The allocation is based on theavailability, capabilities, and connectivity of the multimedia server302. Additional multimedia servers 302 may also be allocated to balancethe load of the currently operating multimedia servers 302 and also tobalance the load on authoritative domains. As part of the user profile,the multimedia server 302 allocation can be predetermined. Similar tocable TV program selection, each user has access to multiple multimediaservers 302 which can deliver content from multiple multimedia sources360. If a multimedia source 360 is served by multiple multimedia servers302, one of the multimedia server 302 can be allocated, for example, byrandom selection.

At step 516, the mobile service platform 340 provides a control channelfrom the mobile service platform 340 to the allocated multimedia server302. This channel is used to direct the allocated multimedia server 302to deliver the multimedia content to the mobile device over an adaptivewireless path. Therefore the request paths from the mobile serviceplatform 340 to the mobile devices is not congested with multimediacontent delivery which is allocated over the plurality of multimediaservers 302 if multiple channels of multimedia content are beingdelivered.

At step 518, if control of the multimedia source is authorized andrequested, the multimedia source provides a logical control path fromthe mobile device through the mobile service platform 340 to themultimedia source 360. In certain application, the logical control pathis directed through the multimedia server 302.

At step 520, the multimedia control command, if provided, is used tocontrol the multimedia source 360. It is possible to terminate therequest with the command if no data delivery is requested. Processingcontinues at step 522, where the mobile service platform 340 obtains amobile device transmission profile 306 to be used by the transcoder 320and the multimedia server 302 for adapting the data transmission to themobile device. In one embodiment, it is assumed that the user isknowledgeable enough to know the download bit rate (i.e. 56 kbps for adial up connection, 19.2 kbps for CDPD, and 100-1000s kbps for WLAN).Alternatively Internet protocol (IP) packets can be pushed to the endmobile client to probe the available bandwidth prior to initiatingdownload of the media stream. The adaptation occurs periodically toupdate the status of the encoding bit rate and frame rate and encodingquantization step size. Initially, the transmission profile provides aconservative estimate of the bit rate.

At step 524, the mobile service platform 340 provides multimedia datadelivery information to the multimedia server 302, and at step 526, themultimedia server 302 invokes multimedia transcoding services based onuser profile 346, device profile 332 and the transmission profile 306.It should be noted that the transcoder 320 location is independent ofthe location of the mobile service platform 340 unless the transcoder320 and the mobile service platform 340 reside on differentauthoritative domains.

At step 528, multimedia source controller 356 instructs multimediasource 360 to transfer the multimedia content to the transcoder 320. Atstep 530, the transcoder 320 transcodes the multimedia content accordingto the device profile 332 and the transmission profile 306. Prior tore-encoding of the multimedia source the request dispatcher 354instructs the transcoder 320 to setup a connection between itself andthe transmitter 304, indicated by a multimedia transmitter IP addressand port number. The transmitter is capable of supporting multipleclients by virtue of allocating each user a unique port number.

At step 532, the request dispatcher 354 instructs the transcoder 320 todeliver the transcoded content to multimedia server 302. At step 534,the transcoder 320 delivers the transcoded content to multimedia client330 for display video or reproduce sound on the mobile device inresponse to the request. The feedback control processor 310, thethroughput manager 336 and the adaptive encoder 320 measure and adaptthe transmission of the multimedia content to current transmissionconditions at step 536.

At step 538, it is determined whether the data delivery request iscomplete. If the data delivery is complete, processing terminates atstep 540, otherwise processing resumes at step 536 where the datatransmission is adapted in response to measured transmissionperformance.

It is understood that the mobile device platform and multimedia serversof the present invention can communicate with a wide variety of mobiledevice types.

One skilled in the art will appreciate further features and advantagesof the invention based on the above-described embodiments. Accordingly,the invention is not to be limited by what has been particularly shownand described, except as indicated by the appended claims. Allpublications and references cited herein are expressly incorporatedherein by reference in their entirety.

1. A system for providing multimedia data to a mobile device, the systemcomprising: a mobile service platform comprising: an authenticationprocessor for authenticating user requests from the mobile device over arequest path; and a multimedia source controller for receiving theauthenticated user requests for controlling at least one multimediasource; at least one multimedia server for receiving control commandsvia a control channel from said mobile service platform, the controlcommands being for controlling delivery of multimedia data to the mobiledevice over a wireless path separate from the request path; and at leastone transcoder for receiving multimedia data from the at least onemultimedia source and coupled to the at least one multimedia server. 2.The system of claim 1 wherein the at least one transcoder is integratedwith a corresponding at least one multimedia server.
 3. The system ofclaim 1 wherein the at least one transcoder is a stand-alone networkdevice coupled between at least one multimedia source and at least onemultimedia server.
 4. The system of claim 1 wherein said mobile serviceplatform further comprises a personalization information managementprocessor for providing personalized data associated with a user of themobile device.
 5. The system of claim 1 wherein the at least onetranscoder comprises an adaptive encoder for encoding multimedia datafor delivery to the mobile device over a wireless path.
 6. The system ofclaim 1 wherein the at least one multimedia server comprises amultimedia transmitter.
 7. The system of claim 6 wherein the mobiledevice communicates with the multimedia transmitter using at least oneof: a CDPD protocol; a TDMA protocol; a GSM protocol; an 802.11protocol; a two way paging protocol; a CDMA protocol; a GPRS protocol; aCDMA-2000 protocol; and a W-CDMA protocol.
 8. The system of claim 1wherein the authentication processor accesses a plurality of userprofile data structures.
 9. The system of claim 1 wherein the personalinformation management processor accesses a plurality of user profiledata structures.
 10. The system of claim 1 wherein the at least onemultimedia server accesses a device profile.
 11. The system of claim 1wherein the at least one multimedia server accesses a transmissionprofile.
 12. The system of claim 1 wherein the multimedia server furthercomprises a feedback control processor coupled to the adaptive encoder,the feedback control processor being for controlling an encoded bit rateof the multimedia data in response to performance measurement feedback.13. The system of claim 12 wherein feedback control processor is furtherfor controlling at least one of a video encoding quantization step size,a video frame rate and video frame size of the multimedia data.
 14. Thesystem of claim 12 wherein feedback control processor is further forcontrolling an encoded bit rate, frame rate and frame size of themultimedia data in response to a transmission profile, a device profileand a measured transmission parameter.
 15. The system of claim 1 whereinthe mobile device comprises a client decoder communicating with the atleast one multimedia server for decoding multimedia data received by themobile device.
 16. A multimedia delivery system comprising; a mobileservice platform having a control path output; a multimedia server forserving multimedia content to a client on a path separate from themobile service platform, the server having a session manager, anencoder, a feedback control and a control path input coupled to thecontrol path output; and a client decoder having a performancemeasurement processor.
 17. The system of claim 16 wherein the multimediaserver further comprises a decoder coupled to a plurality of multimediasources.
 18. The system of claim 16 wherein the multimedia serverfurther comprises a network state information data structure coupled tothe feedback control and decision processor.
 19. A wireless network,comprising: a mobile service platform having an authentication processorfor authenticating user requests from the mobile device and a multimediasource controller for receiving the authenticated user requests forcontrolling at least one multimedia source; a multimedia server forreceiving control commands from said mobile service platform and coupledto an adaptive transcoder for wireless data delivery; and a clientdecoder coupled to said mobile service platform via a user request pathand coupled to said multimedia server via a wireless data delivery pathseparate from the user request path.